Applying Information Retrieval Techniques to Event Log Analysis for Intrusion Detection

This paper explores the application of probabilistic information retrieval theories to the field of log analysis and host-based intrusion detection. Strong similarities exist between intrusion detection and information retrieval. Using information retrieval techniques may yield significant improvements to the performance of intrusion detection systems. This paper provides a brief review of current relevant research in intrusion detection and log analysis, introduces information retrieval methods appropriate for intrusion detection, and proposes a framework for an experimental log analysis system. The proposed system is based on Bayesian probability theory and uses a term frequency-inverse document frequency (TF-IDF) measure to identify anomalies.